Bellman Equations for Stochastic Programs
نویسنده
چکیده
This is a revised version of a paper entitled \World Modeling vs. World Axiomatizing" which accompanied an invited talk at LPNMR-99. Abstract. This paper gives a model-checking style algorithm for verifying the decision-theoretic utility of a handwritten robot controller. The paper presents a simple Turing-complete programming language for expressing robot control programs. The robot control language supports stochastic nondeterminism, complex data structures, conditional expressions , and recursive procedures. A similar Turing-complete programming language for writing models of the robot's environment is also given. The paper shows that if a given controller running in a given environment has only nitely many behaviors (as deened in the paper) then the value of that controller in that environment can be computed by iterating appropriately generalized Bellman equations.
منابع مشابه
Bellman Equations Associated to The Optimal Feedback Control of Stochastic Navier-Stokes Equations
We study the infinite dimensional second-order Hamilton-Jacobi-Bellman equations associated to the feedback synthesis of stochastic Navier-Stokes equation forced by space-time white noise. Uniqueness of viscosity solutions and short-time existence are proven for these infinite dimensional partial differential equations.
متن کاملLinear-exponential-quadratic Gaussian Control for Stochastic Partial Differential Equations
In this paper a control problem for a controlled linear stochastic equation in a Hilbert space and an exponential quadratic cost functional of the state and the control is formulated and solved. The stochastic equation can model a variety of stochastic partial differential equations with the control restricted to the boundary or to discrete points in the domain. The solution method does not req...
متن کاملHamilton-Jacobi-Bellman Equations and the Optimal Control of Stochastic Systems
In many applications (engineering, management, economy) one is led to control problems for stochastic systems : more precisely the state of the system is assumed to be described by the solution of stochastic differential equations and the control enters the coefficients of the equation. Using the dynamic programming principle E. Bellman [6] explained why, at least heuristically, the optimal cos...
متن کاملSecond Order Hamilton--Jacobi Equations in Hilbert Spaces and Stochastic Boundary Control
The paper is concerned with fully nonlinear second order Hamilton{Jacobi{Bellman{ Isaacs equations of elliptic type in separable Hilbert spaces which have unbounded rst and second order terms. The viscosity solution approach is adapted to the equations under consideration and the existence and uniqueness of viscosity solutions is proved. A stochastic optimal control problem driven by a paraboli...
متن کاملSolvability of Forward-Backward SDEs and the Nodal Set of Hamilton-Jacobi-Bellman Equations
Abstract. In this paper, the solvability of a class of forward-backward stochastic di erential equations (SDEs for short) over an arbitrarily prescribed time duration is studied. We design a stochastic relaxed control problem, with both drift and di usion all being controlled, so that the solvability problem is converted to a problem of nding the nodal set of the viscosity solution to a certain...
متن کامل